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"IMAGE VIEWING SYSTEM AND METHOD FOR GENERATING FILTERS FOR FILTERING IMAGE 
FEATURES ACCORDING TO THEIR ORIENTATION" 

Description 

Field of the Invention 

The invention relates to an Image processing system and an image processing method to 
generate adaptive filters, whose supports are oriented according to focal orientations of features of 
interest in an image to be processed. The invention more particularly relates to such processing 
system and method that are applied to medical images. The invention also relates to a medical 
examination apparatus having such an image processing system and means for display. The invention 
finds an application in the industry of image processing systems and medical examination 
apparatuses. 

Background of the Invention 

A method to smooth and filter images is disclosed in the publication "A Framework for 
Anisotropic Adaptive Filtering and Analysis of Sequences and volume" in IEEE International 
Conference on Acoustics, Speech and Signal Processing (ICASP); San Francisco, March 23-26, 1992 ; 
p.469-472, vol.3, by H. Khutsson, L Haglund and G.H. Granlund. The approach described in the cited 
publication makes use of quadrature and steerable filters having a set of different orientations 
including at least three different orientations in the case of 2-D image processing, and at least six 
different orientations in the case of 3-D image processing. The algorithm written from this approach 
requires the application of all the filters involved in the filtering of one image. The resulting algorithm 
complexity has been found to be high. So, the calculation time is particularly long. 

Summary of the Invention 

The present invention has for an object to provide an image viewing system having 
processing means to generate filters whose supports are oriented according to local orientations of 
features of interest in an image to be processed. In particular, it is an object of the invention to 
provide an image viewing system having analysis means for analyzing the content of a noisy image 
representing various objects in order to determine the direction of features of interest in an image; 
and having synthesis means for synthesizing filters whose supports are oriented for filtering noise or 
enhancing features in the image. 

It is specifically an object of the invention to provide a system for generating such oriented 
filters whose supports do not present approximated shapes but instead present exact shapes that do 
not vary according to their angle of orientation. It is also an object of the invention to provide a 
system using very low complexity image processing techniques for generating such oriented filters. 
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Such an image viewing system Is claimed in claim 1. 

The system of the invention has very powerful means to constitute oriented filters. An 
advantage of the system of the invention is that the analysis means shows very low complexity of 
calculation, resulting in substantial computation saving. In addition, the synthesis means of the 
system of the invention can reproduce the exact shapes of some typically targeted enhancement 
filters, such as for example oriented Gaussian kernels. Besides, the computations means according to 
the invention can be efficiently implemented using very simple techniques. An other advantage of the 
system of the Invention Is that the required computation time is especially short. The system of the 
invention has means to constitute a very fast and general D-dimensional adaptive filter. 

The system of the invention can be implemented so that it is minimally demanding In terms 
of input/output, thus making a very good filter candidate for general purpose processor 
implementation. In fact, the system of the invention presents the advantage that it does not require 
any memory means. Since at the present time, processing means are very efficient for producing 
results of calculations, but are not very efficient for extracting data from memories, hence, the system 
of the invention, which only comprises calculation means, is very efficient. 

In advantageous embodiments of the invention, the user may dispose of control means for 
interacting with the analysis means to select features of interest in the image and/or with the 
synthesis means to select shapes of the filter supports. 

The invention may refer to image data in which each data point (pixel or voxel) is attributed 
a single scalar intensity value. This can be extended to multi-spectral data in which several values are 
attributed to each data point For example, in colored images, one may refer to Red, Green and Blue 
components. Multidimensional data can be dealt with, such as one dimension, spatial 2D, spatial 3D, 
spatial multidimensional data plus time data. When one of the dimension is time and when one aims 
at real-time results, the method of the invention may have recursive steps for using data recorded at 
previous instants. 

Brief Description of the Figures 

The invention is described hereafter in detail in reference to diagrammatic figures, wherein : 
FIG.l is a functional block diagram illustrating the means of the image viewing system; 
FIG.2A, FIG.2B and FIG.2C illustrate the generation of site vectors; 

FIG.3A represents a Gaussian weighting function fa FIG.3B represents a Laplaclan weighting 
function f^ FIG.3C represents a set of 2-D oriented kernels based on Gaussian and Laplacian 
weighting functions; 

FIG.4 illustrates the steps carried out by the viewing system of the invention; 

FIG.5 is a functional block diagram of a viewing system and a medical apparatus according 
to the invention. 



3 



PHFR030002 EPp 



Description of Embodiments 

The invention relates to an image viewing system having processing means to generate 
filters whose supports are oriented according to local orientations of features of interest in an image 
to be processed. The invention relates to such a system for adaptive filtering of multidimensional 
signals for quality improvement application including noise reduction and the enhancement of valuable 
details. This system has processing means for achieving this goal in two steps: first means for signal 
analysis and, using the results of this analysis, second means for filter synthesis. In particular, the 
invention relates to an image viewing system having analysis means for analyzing the content of a 
noisy image representing various objects, in order to determine tine direction of features of interest in 
an image; and having synthesis means for synthesizing filters with oriented supports apt to filter noise 
or enhance features in the image. 

Specifically, the invention relates to an image processing system using analysis means that 
provides orientation of features of interest directly; and synthesis means for generating oriented filters 
having supports that do not present approximated shapes but instead exact shapes, which remain 
unchanged when their angles of orientation vary. 

Besides, the invention has for an object to provide such a system that produce the results 
within short computation time using processing means that are simple to implement. 

The system comprises means of acquisition of image data including point coordinates, * 
denoted by x, and an intensity value, denoted by I(x), for each image point x. The proposed system 
deals with a multi-dimenstonal image, whose dimension is denoted by D, It is not exduded that D •= 
2. For a D-dimensional image, each data point is defined by a D-tuple point x associated to 
coordinates of the form (Xu x^.-Xp) where each of the components x u X2,...Xo varies in a predefined 
range, for instance 0<Xi<Ii ; 0<x 2 <I 2 ; 0 <Xd < Id- To each such D-tuple, corresponds an image 
value, which may be a single scalar value. In the case of a monochromatic image, the scalar value is 
the luminance intensity. For a color image, the image value may be represented by a set of three 
intensities representing the Red, Green and Blue components. 

FIG.l is a block diagram of the image processing system of the invention. Referring to 
RG.l, the input image data I(x) are processed using the following processing means: 

1) Analysis means for analyzing the content of the original image. 

The analysis means comprises a Direction Estimator 10 that receives the input image data 
I(x) referred to by 1 and that outputs, on a pixel basis, a set of multidimensional orthogonal unit 
vectors denoted by (e^ ...,e d ) T where d is the input signal dimension. These vectors are aligned with a 
locally most prominent feature. 

The nature of the feature on which direction should be estimated is user selectable. The 
user may interact for feature selection using the user control interface 158. For instance, the user 
visualizes the input image and selects in the image a feature of interest that may be an edge or a 
border of an object. Once the user has chosen a feature of interest, the system of the invention has 
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means to determine the orientation angle of the feature of interest. Hence at each point of the image, 
the analysis means provides a corresponding set of d-dimensional orthogonal unit vectors. The 
Direction Estimator 10 may use any technique known to those skilled in the art to determine the 
direction of the feature of interest. This technique may comprise gradient estimation (first order 
derivatives) or local curvature estimation (second order derivatives) or eigen values of the tensor of 
structure. There are numerous possible techniques of direction estimation. 

The analysis means further comprises a Site Generator 20, which produces a set of local 
vectors r spanning a filter window around a reference point. Referring to FIG.2A, FIG.2B and FTG.2C, 
a small window, which comprises a few number n of points, is generated around a reference point, 
denoted by O , such as the center of the window. The window is a filter support that comprises the 
ensemble of points, which are used to calculate the output of the filter at the current point (x) of the 
Image. For example in 2-D, the shape of the support may be a square, as shown In FIG.2A, or a circle, 
as shown in FIG.2B and RG.2C, and the reference point O may be the center of the square or the 
circle. The Site Generator 20 generates, for each current point (x) a set of n site vectors r, which is 
used to form a neighborhood N(x) around the current point (x). In FIG.2A to 2C, only few vectors r 
are represented for simplicity. The user may interact, through the user control interface 158, in order 
to chose the shape and area of the filter support for the Site Generator 20. The set of n site vectors r 
of the neighborhood N(x) produced by the Site Generator 20 is passed on each point x of the image 
I(x) to be filtered. 

Now, to a given current point x of the d-dimensional image, correspond a set of d 
orthogonal unit vectors denoted by (ei, ei,...,e d ) T forming a d-dimensional orthogonal vector basis and 
a set of n site vectors denoted by r in the neighborhood N(x). 

2) Synthesis means for synthesizing oriented filters. 

The synthesis means comprises a scalar product unitoutputting scalar products <cfe r> 
referred to by 12.1 to 12.d. for each image point, the scalar product unit performs the product 
between each site vector r with ad the vectors e x to e d of the d-dimensional orthogonal basis. 

The synthesis means further comprises a set of number d of elementary weight scalar 
functions f t to U, which transforms the scalar products into elementary weight coefficients. So, there 
is one function fj( ) per vector e|. Each scalar product <e^ r> is regarded as the Input of a scalar 
function fj with (1 £ i £ d). The elementary weight scalar functions f x to f d , which receive the scalar 
values <ej, r>, produce a number d of scalar weighting coefficients W x to W d at their output. 

The content of those scalar functions directly depends on the nature and amount of 
enhancement the user wants to apply on the selected feature. The nature of features and the amount 
of enhancement, which the user may want to apply on a given set of selected features, are user 
selectable. The nature of features and the amount of enhancement can be associated to predefined 
functions ft. The functions ft can be pre-loaded In selectable Tables". The user does not have to 
himself provide the functions ft. Instead, the scalar functions can be selected from a predefined 
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library. The user can pinpoint one "Table" of the library, each "Table" corresponding to a predefined 
'Taste" of the user. The user may select or modify the nature of the elementary weight scalar 
functions fi to f d . To this end the user acts on the control means 158. The nature of the elementary 
weight scalar functions determines the nature of image processing that will be applied according to 
local directions determined by the orthogonal d-dimensional basis. If the user selects scalar function 
that always gives positive results, then smoothing filters will be generated. Instead, if the user selects 
scalar function that sometimes gives negative results, then enhancing filters can be generated. 

For example, referring to FIG.3A, where d=2 in a 2-D image, a Gaussian scalar weighting 
coefficient Wi(r)= fi(p) with p = <e u r>, may be chosen on the vector of direction ei. And then, 
referring to FIG.3B, in said 2-D image, a Lapladan scalar weighting coefficient W 2 (r)= f 2 (p) with p = 
<e2/ r>, may be chosen on the vector of direction e* this Lapladan weighting coeffident W 2 being a 
symmetrical function outputting positive values inside a canal of abscissae near zero and producing 
negative values for the abscissae each side of the canal. 

The synthesis means further comprises a combination unit 15 denoted by n, which 
combines the number d of scalar weighting parameters Wi to W d . The combination unit gathers the 
elementary weights f| (<d, r>) into a single scalar value W(r). In a preferred embodiment, the 
combination unit n may be a product unit, which performs a simple d-term product between the 
scalar weighting parameters W x to W d and which issues the weight coeffidents W(r) of the site rAhat 
will be used to form adaptive oriented filter kernels in the filter to be generated. 

Now, to each point of the image correspond a number n of sites and a number n of weight 
coeffidents W(r), referred to by 31, each weight corresponding to a given site vector r. The filter- 
coefficients w(r) are generated on the fly, which allows to filtering features in all kinds of directiohs. 

3) Filter generating means. 

The filter generation means are formed by a filtering function g, which may of the form 
given by the following formula: 

ZW(r)I(x + r) 

a = l (1) 

r 

According to this example, the filtering generation means are apt to generate a filter 
produdng a normalized local weighted sum, which Is performed at all the input pixels (x+r) of 
intensity I(x+r), within the neighborhood N(x), by the W(r) weights. Normalization is used to preserve 
the average of the signals. The normalization corresponds to divide the above weighted sum by the 
sum of the W(r) weights over the support S of the site neighborhood N(x). 

For example, referring to FLG.3C, using the scalar weighting coeffidents Wiand as 
illustrated by FIG.3A and FIG.3B, oriented filter kernels 31 are generated. A smoothing filter has a 
support length along ei that depends on the standard deviation value or of tine Gaussian scalar fu from 
which the scalar weighting coeffident W x is calculated, and a support width along ez that depends on 
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the width of the canal of the function f* from which the scalar weighting coefficient W 2 is calculated,. 
The negative coefficients of W 2 in the portions disposed each side of the canal determine the strength 
of the enhancement provided by the filter generated according to the invention. 

As illustrated by FIG.3C, this operation of summing 15 is apt to produce an exact shape for 
the filter kernels 31, which are used to filter oriented features in the image. According to the 
invention, the filter kernels 31 are oriented along the current estimated directions e x to ed. Besides, 
the filter kernels show the exact shapes produced by the use of the computed scalar weighting 
coefficients Wj and W2, which shapes are invariant whatever the filtering orientation angle. 

Since the filter coefficients W(r) are generated on the fly, the synthesizing means does not 
generate specific filter banks of kernels with predefine sets of directions- Instead, the filtering means 
of the invention allows to filtering features in all kinds of directions. 

The invariance of the filter support shapes in function of the filtering orientation angles is a 
unique, very important and very advantageous result of the invention. 

In the case when the image intensities are multicomponent signals or vectorial, the 
combination formula (1) Is still valid. This formula yields a weighted sum of vectors, in the case when 
all the signal components have the same directions. For instance, in the case of a signal comprising 
red, blue, green components, said signal components must have the same direction. When said signal 
components do not show the same directions, in that case, the filter of the invention may be used 
independently on each of the components of the signal. One direction is estimated for each signal 
component and one set of weighting coefficients W(r) is further estimated for each component and 
each image point. Then, formula (1) Is applied for each signal component independently. 

4) Advantages. 

This image processing system is able to produce the exact form of some typically targeted 
enhancement filters Including oriented Gaussian kernels, as illustrated by RG.3A to 3C. In addition, it 
constitutes a very fast and general d-dimensional adaptive filtering system. Furthermore, the system 
can be implemented so that it becomes minimally demanding in terms of input/output, thus making a 
very good filter candidate for general purpose processor implementation. 

This system is not demanding any complexity regarding the choice of the direction 
estimation means. The choice is up to the user. The direction estimation means may be based on 
simple gradient calculation or on more elaborate calculation of eigen vectors of tensors of structure or 
on Heissen functions giving the orientation of principal curvatures. 

The filter kernels W(r) are generated on the fly, which allows to filtering features in all kinds 
of directions. There is no filter bank with predefine sets of directions. 

The system uses principally scalar functions and scalar products, which are particularly 
simple to implement The use of scalar products produces particularly rapid results, since they may be 
estimated in an incremental manner with a simple supplementary addition for each further product 
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The weighting scalar functions may be simply tabulated, using very small tables since they 
are only applied to the dimensions related to the dimensions of the filter used in the site generation. 
In fact, the scalar products f } (<e, . r>) are inferior to the radius of said filter support. Hence, the 
tabulated functions are applied to the few points corresponding to the very small support 

The function of combination n may be very simple, especially In the case when d = 2 and n 
is a product, thus being a product of two terms. After having applied this simple function of 
combination n, the next operation is a simple normalized weighted sum. 

Hence, the total cost of filter generation and filtering operation according to the invention is 
extremely low. This low cost does not prevent the system of the invention to produce very interesting 
shapes of filter kernels, provided that appropriate weighting function fj are chosen, such as illustrated 
by FIG.3Ato3C. 

The filter generation means according to the invention does not require any intermediary 
memory means. All the computations are performed in a continuous way. Since processors are very 
efficient in the computing steps and show low efficiency in extracting data from memory, the system 
of the invention is very appropriate to be associated to computers. 

The filter of the invention may be applied in the spatial domain. However, since it may be 
applied to any number d of dimensions and since no specific technique for determining directionls 
imposed, then it may be applied in spectral domain to transformation coefficients such as FFT, DCT 
Hermite transforms, Hermite wavelets etc. or on sub-bands of Gaussian or Laplacian pyramid. « 

5) Image processing method. 

Referring to FIG.4, the image processing method, for generating a multidimensional 
adaptive oriented filter to process image data in a number d of dimensions, operated by the above- 
described system comprises steps of: 

computing, for each point of the image, a number d of scalar products, <ei.r> to 
<e<j.r>, of a number d of vectors, ei to of an oriented vectors basis by a number n of site vectors 
r defined on a neighborhood N(x) of the point, said neighborhood including n points. The scalar 
products are respectively weighted by a number d of weighting functions, fi to f^ to produce a 
number d of weighted scalar coefficients Wi(r) to W d (r); 

combining in step II the weighted scalar coefficients Wi(r) to W d (r) for producing a 
set of one-scalar weight coefficients W(r) over said neighborhood N(x) that define adaptive oriented 
filter kernels [w(r)]; and 

filtering the image data by a function g for producing filtered image data g(x) from 
the combination of the image data I(x+r), defined over the neighborhood N(x), with the set of one- 
scalar weight coefficients W(r) forming the adaptive oriented filter kernels [w(r)]. 
This method particularly comprises steps of: 

direction estimation 10 for providing, at each image point, an oriented orthogonal 
vector basis of a number d of vectors e x to 
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a site vector generation 20 for providing n site vectors over a local vector support, 
which Is formed by a neighborhood N(x) of n points around each image point; and 

computation of d scalar products <q . r> of the vectors e x to e d of the orthogonal 
vector basis by the n site vectors r; and weighting the scalar products <ei . r> through a number d of 
scalar weighting functions h to f d . 

According to this method, the combination means n for producing a one-scalar weight 
coefficients W(r) is a d-terms product; the vectors of the oriented basis of vectors are produced by a 
direction estimation step 10 that includes estimation of directions based on gradient estimation or 
eigen vectors of Hessien or of tensors of structure; the weighting functions comprise Gaussian 
functions and/or symmetrical functions whose output is positive for values near zero inside a canal 
and is negative each side of the canal beyond the canal. 

6) Medical viewing system and apparatus 

The above-described means and steps can be implemented in the viewing system of the 
invention. Fig,5 shows the basic components of an embodiment of an image viewing system in 
accordance to the present invention, incorporated in a medical examination apparatus. The medical 
examination apparatus 151 may include a bed 110 on which the patient lies or another element for 
localizing the patient relative to the imaging apparatus. The medical imaging apparatus 151 may be a 
CT scanner or other medical imaging apparatus such as x-rays or ultrasound apparatus. The image 
data produced by the apparatus 151 is fed to data processing means 153, such as a general-purpose 
computer, that carries out the steps of the method. The data processing means 153 is typically 
associated with a visualization device, such as a monitor 154, and an input device 155, such as a 
keyboard, or a mouse 156, pointing device, etc. operative by the user so that he can interact with the 
system. The data processing device 153 is programmed to implement a method of processing medical 
image data according to invention. In particular, the data processing device 153 has computing means 
and memory means to perform the steps of the method. A computer program product having pre- 
programmed instructions to carry out the method may also be implemented. 

The drawings and their description herein before illustrate rather than limit the invention. It 
will be evident that there are numerous alternatives that fall within the scope of the appended claims. 
Moreover, although the present invention has been described in terms of generating image data for 
display, the present invention is intended to cover substantially any form of visualization of the image 
data including, but not limited to, display on a display device, and printing. Any reference sign In a 
claim should not be construed as limiting the claim. 
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Claims 

1. Image processing system for generating a multidimensional adaptive oriented filter to be 
applied to the point intensities of an image formed in a number d of dimensions, comprising: 

processing means for producing, from the image point intensities [I(x)], adaptive oriented 
filter coefficients [w(r)] formed through combination (n) of weighted scalar coefficients 
[Wi(r)..Wi(r)...w d (r)], which coefficients are weighted scalar products [w,=f,(<e I . r>)] of a number d 
of vectors (e x ...e,...ed) of an oriented vectors basis, by a number n of local vectors (r) estimated over . 
a neighborhood [N(x)] around the current image point 

2. Image processing system of Claim 1, comprising: 

product means for producing the weighted scalar coefficients EWi(r),..,Wi(r),..,W d (r)]; 

combination means (n) for combining the weighted scalar coefficients to produce a set of 
one-scalar weight coefficients [W(r)] forming the adaptive oriented filter kernels; and 

filtering means (g) for producing filtered image data [g(x)3 from the combination of the 
image data [I(x+r)] over the neighborhood [N(x)] with the one-scalar weight coefficients [W(r)]. 

3. The system of one of Claims 1 or 2, comprising: f 

a direction estimator (10) for providing, at each image point, an oriented orthogonal basis of 
a number d of vectors (ei-ea); 

a site generator (20) for providing n site vectors forming a neighborhood [N(x)] of the *"< 
image points; and 

product means for computing a number d of scalar products [<ei . r>] of the vectors of the 
orthogonal vector basis for each of the n site vectors (r), for each image point. 

4. The system of one of Claims 1 to 3, comprising: 

weighting means for weighting the scalar products [<e f . r>] through scalar functions (f|). 

5. The system of one of Claims 1 to 4, comprising: 

filtering means including a weighted normalized sum of the products of image data [I(x+r)] 
over the neighborhood [N(x)] of the image points by the one-scalar weight coefficients [W(r)] forming 
the adaptive oriented filter kernels. 

6. The system of one of Claims 1 to 5, wherein: 

the combination means (n), for producing the set of one-scalar weight coefficients [W(r)] 
forming the adaptive oriented filter kernels, is a d-terms product 
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7. The system of one of Claims 1 to 6, wherein: 

the direction estimator (10) for producing the vectors of the oriented vector basis includes 
direction estimation means for estimating direction of image features based on gradient estimation or 
eigen vectors of Hessien or of tensors of structure. 

8. The system of one of Claims 1 to 7, comprising: 

weighting means for producing the weighting functions chosen among Gaussian functions 
and/or symmetrical functions whose output is positive for values near zero inside a canal and is 
negative each side of the canal beyond the canal. 

9. The system of one of Claims 1 to 8, comprising control means for the user to select image 
oriented features to be processed through the direction estimator (10) and/or the type of 
neighborhood [N(x)] for the Site Generator (20). 

10. The system of one of Claims 1 to 9, comprising control means for the user to choose the 
shape of the weighting functions (fi), for selecting the amount of filtering, which corresponds to 
positive coefficients, and for selecting the amount of enhancement, which corresponds to negative 
coefficients. 

11. Image processing method for generating a multidimensional adaptive oriented filter to 
process image data in a number d of dimensions, using a system as Claimed in one of Claims 1 to 10, 
comprising steps of: 

computing, from the image point intensities p(x)], adaptive oriented filter kernels [w(r)] 
formed through the combination (II) of weighted scalar coefficients [Wi(r)..W|(r)...W d (r)], which 
coefficients are weighted scalar products [Wi=fi(<ej . r>)] of a number d of vectors of an oriented 
vectors basis (ei...ei...e d )/ by a number n of focal vectors (r) estimated over a neighborhood [N(x)] 
around the current image point;. 

filtering (g) the image data P(x+r)] by the adaptive oriented filter kernels [w(r)] for 
producing filtered image data [g(x)] over the neighborhood [N(x)]. 

12. Medical examination apparatus comprising means to acquire d-dimensional image data 
[I(x)], a system as Claimed in one of Claims 1 to 10 and further comprising a display system (154) for 
visualizing processed images and user control means (158) for selecting weighting functions and/ or 
acting on the direction estimator and/or the site generator. 

13. A computer program product comprising a set of instructions for carrying out the method as 
claimed in Claim 11. 
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Abstract 

Image processing system for generating a multidimensional adaptive oriented filter to 
process image data in a number d of dimensions, comprising product means for producing weighted 
scalar coefficients [Wi(r),..,W,(r), ..,W d (r)] of a number d of vectors of an oriented basis of vectors by 
a number n of local vectors related to each point; combining means (n) for producing a set of one- 
scalar weight coefficients [W(r)] from the combination of the weighted scalar products; and filtering 
means (g) for producing filtered image data [g(x)] from the combination of the image data p(x)] with 
the one-scalar weight coefficients [W(r)]. The system further comprises a direction estimator (10) for 
providing, at each image point, an oriented orthogonal basis of a number d of vectors (e^.e,,..^); a 
site generator (20) for providing n site vectors of a local vector support; and product means for 
computing d scalar products of vectors of the orthogonal basis by each of the n site vectors. This 
system may also comprise means for providing weighting means for the scalar products through . 
scalar functions. The filtering means may comprise a weighted normalized sum of the image data by 
the one-scalar weight coefficients [W(r)]. 
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